Papers with model transfer
TransAdv: A Translation-based Adversarial Learning Framework for Zero-Resource Cross-Lingual Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition are limited by noise in translation . Existing approaches to named entities recognition are mainly based on labeled data . |
| Approach: | They propose a framework to mitigate lexical and syntactic errors of translated data . they propose to use multi-level adversarial learning and multi-model knowledge distillation to mitigate noise . |
| Outcome: | The proposed framework mitigates lexical and syntactic errors of translated data . it achieves competitive performance to state-of-the-art models . |
On Dataset Transferability in Active Learning for Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) aims to reduce labeling costs by querying the examples most beneficial for model learning. |
| Approach: | They propose to query examples most beneficial for model learning by querying data points most informative for labeling. |
| Outcome: | The proposed method reduces labeling costs by querying the examples most beneficial for model learning. |
Basque and Spanish Counter Narrative Generation: Data Creation and Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Davidson et al.: hate speech is a growing media presence, but research on generating CNs has been limited . he says a new dataset for CN generation is available for basque and spanish . this dataset is based on a multilingual encoder-decoder model . |
| Approach: | They propose a new Basque and Spanish dataset for automatic CN generation . they use machine translation and professional post-edition to generate CNs in both languages . |
| Outcome: | The proposed datasets show that training on post-edited data improves generation over monolingual settings . similar results in zero-shot crosslingual evaluations show multilingual data augmentation outperforms training in English and Spanish . |
Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have proposed data-based cross-lingual transfer as an effective technique for cross-linguistic sequence labelling, but they have failed to perform well. |
| Approach: | They propose to use data-based cross-lingual transfer to train supervised models from a source language to unlabelled target languages. |
| Outcome: | The proposed techniques outperform data-based cross-lingual transfer approaches in a zero-shot setting. |
Learning Disentangled Representations of Texts with Application to Biomedical Abstracts (D18-1)
Copied to clipboard
| Challenge: | a method for learning disentangled representations of texts that encode distinct and complementary aspects is proposed . a classic problem in distributed representation learning is that it is difficult to determine what information individual dimensions encode. |
| Approach: | They propose a method for learning disentangled representations of texts that encode distinct and complementary aspects by a adversarial objective based on the (dis)similarity between triplets of documents with respect to specific aspects. |
| Outcome: | The proposed method can be used to perform aspect-specific retrieval on biomedical abstracts. |
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Deep neural networks excel at learning from labeled data, but learning from unlabeled data remains a challenge. |
| Approach: | They review neural unsupervised domain adaptation techniques which do not require labeled target domain data. |
| Outcome: | The proposed techniques are more challenging yet widely applicable. |
To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to select transfer sources are limited by text and task similarity, which limits their application in transfer settings where both the task and the text domain change. |
| Approach: | They propose a model similarity measure that represents text and task similarity jointly to automatically determine which and how many sources to exploit. |
| Outcome: | The proposed approach improves performance by 24 F1 points for predicting promising sources across domains and tasks with similar models. |
Chinese Opinion Role Labeling with Corpus Translation: A Pivot Study (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlike most of the previous work focusing on the English language, this paper focuses on the Chinese ORL task. |
| Approach: | They propose to use a standard English MPQA dataset to construct a Chinese ORL dataset and investigate the effectiveness of cross-lingual transfer methods. |
| Outcome: | The proposed method is able to detect and improve the performance of the proposed method in Chinese. |
Multi-Task Transfer Matters During Instruction-Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Instruction-tuning improves a model’s ability to learn in-context, but the mechanisms that drive in-constext learning are poorly understood. |
| Approach: | They propose to train a model on hundreds of tasks to improve its ability to learn in-context. |
| Outcome: | The proposed methods improve model transfer and in-context generalization, suggesting catastrophic forgetting may impact in-constext learning. |
MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing zero-shot dialogue state tracking models suffer from domain transferring and partial prediction problems. |
| Approach: | They propose to establish connections between similar slots in different domains to improve model transfer performance in unseen domains. |
| Outcome: | Empirical results show that the proposed model achieves the goal accuracy of 57.13% on MultiWOZ2.1 and 55.4. |
Adaptive Instruction Composition for Automated LLM Red-Teaming (2026.acl-long)
Copied to clipboard
| Challenge: | Adaptive instruction composition is a framework for red-teaming that combines crowdsourced texts with random combinations to optimize effectiveness and diversity. |
| Approach: | They propose a framework that combines crowdsourced texts according to an adaptive mechanism trained to optimize effectiveness with diversity. |
| Outcome: | The proposed framework outperforms random combination on effectiveness and diversity metrics even under model transfer. |